Método adaptativo en tiempo real para la detección de anomalías mediante aprendizaje automático

[Resumen] La detección de anomalías es la subrama del aprendizaje automático encargada de construir modelos capaces de diferenciar entre datos normales y anómalos. Ya que los datos normales son los que prevalecen en estos escenarios y sus características suelen ser bien conocidas, el entrenamiento de los sistemas se realiza mayormente mediante estos conjuntos de datos normales, lo que diferencia a la detección de anomalías de otros problemas de clasificación estándar. Debido al habitual uso de estos sistemas en monitorización y a la inexistencia de métodos capaces de aprender en tiempo real, en este proyecto de investigación se presenta un nuevo método que proporciona dicha capacidad de adaptación online. El método desarrollado recibe el nombre de OnlineS-DSCH (Online and Subdivisible Distributed Scaled Convex Hull) y basa su funcionamiento en las propiedades de los cierres convexos. Tras su desarrollo, se ha evaluado y comparado su rendimiento con los principales algoritmos del área sobre diferentes conjuntos de datos, tanto reales como artificiales. Como consecuencia, se ha obtenido un algoritmo con la capacidad de aprendizaje online, fácilmente configurable y cuyas predicciones son justificables, todo ello sin que suponga una merma en su eficacia en relación a las otras soluciones disponibles. Por último, su ejecución se puede llevar a cabo de manera distribuida y en paralelo, lo que supone una ventaja interesante en el tratamiento de conjuntos de datos de alta dimensionalidad.[Abstract] In machine learning, anomaly detection is the branch responsible for building models capable of differentiating between normal and anomalous data. Normal data prevail in scenarios of this type, and their features are usually well known, so the training phase is largely done through these normal data sets, which differentiates the anomaly detection from other standard classification problems. Due to the usual use of these systems in monitoring and the lack of methods capable of learning in real time, this research project presents a new method that provides such online adaptability. The method developed is called OnlineS-DSCH (Online and Subdivisible Distributed Scaled Convex Hull) and bases its operation on the properties of convex hulls. After its development, its performance has been evaluated and compared with the main algorithms of the area on different real and artificial data sets. As a consequence, an algorithm with the online learning capacity, easily configurable and whose predictions are justifiable, has been obtained, all without diminishing its effectiveness in relation to the other available solutions. Finally, its execution can be carried out in a distributed and parallel way, which is an interesting advantage in the treatment of high dimensionality data sets.Traballo fin de grao (UDC.FIC). Enxeñaría informática. Curso 2018/201

Adaptive Real-Time Method for Anomaly Detection Using Machine Learning

[Abstract] Anomaly detection is a sub-area of machine learning that deals with the development of methods to distinguish among normal and anomalous data. Due to the frequent use of anomaly-detection systems in monitoring and the lack of methods capable of learning in real time, this research presents a new method that provides such online adaptability. The method bases its operation on the properties of scaled convex hulls. It begins building a convex hull, using a minimum set of data, that is adapted and subdivided along time to accurately fit the boundary of the normal class data. The model has online learning ability and its execution can be carried out in a distributed and parallel way, all of them interesting advantages when dealing with big datasets. The method has been compared to other state-of-the-art algorithms demonstrating its effectiveness.This work has been supported by Spanish Government’s Secretaría de Estado de Investigación (Grant TIN2015-65069-C2-1-R), Xunta de Galicia (Grants ED431C 2018/34 and ED431G/01) and EU FEDER funds.Xunta de Galicia; ED431C 2018/34Xunta de Galicia; ED431G/0

Fast deep autoencoder for federated learning

Financiado para publicación en acceso aberto: Universidade da Coruña/CISUG[Abstract]: This paper presents a novel, fast and privacy preserving implementation of deep autoencoders. DAEF (Deep AutoEncoder for Federated learning), unlike traditional neural networks, trains a deep autoencoder network in a non-iterative way, which drastically reduces training time. Training can be performed incrementally, in parallel and distributed and, thanks to its mathematical formulation, the information to be exchanged does not endanger the privacy of the training data. The method has been evaluated and compared with other state-of-the-art autoencoders, showing interesting results in terms of accuracy, speed and use of available resources. This makes DAEF a valid method for edge computing and federated learning, in addition to other classic machine learning scenarios.This work was supported in part by grant Machine Learning on the Edge - Ayudas Fundación BBVA a Equipos de Investigación Científica 2019; the Spanish National Plan for Scientific and Technical Research and Innovation (PID2019-109238GB-C22 and TED2021-130599A-I00); the Xunta de Galicia (ED431C 2022/44) and ERDF funds. CITIC is funded by Xunta de Galicia and ERDF funds. Funding for open access charge: Universidade da Coruña/CISUG.Xunta de Galicia; ED431C 2022/4

A One-Class Classification method based on Expanded Non-Convex Hulls

[Abstract]: This paper presents an intuitive, robust and efficient One-Class Classification algorithm. The method developed is called OCENCH (One-class Classification via Expanded Non-Convex Hulls) and bases its operation on the construction of subdivisible and expandable non-convex hulls to represent the target class. The method begins by reducing the dimensionality of the data to two-dimensional spaces using random projections. After that, an iterative process based on Delaunay triangulations is applied to these spaces to obtain simple polygons that characterizes the non-convex shape of the normal class data. In addition, the method subdivides the non-convex hulls to represent separate regions in space if necessary. The method has been evaluated and compared to several main algorithms of the field using real data sets. In contrast to other methods, OCENCH can deal with non-convex and disjointed shapes. Finally, its execution can be carried out in a parallel way, which is interesting to reduce the execution time

Fast Deep Autoencoder for Federated learning

This paper presents a novel, fast and privacy preserving implementation of deep autoencoders. DAEF (Deep Autoencoder for Federated learning), unlike traditional neural networks, trains a deep autoencoder network in a non-iterative way, which drastically reduces its training time. Its training can be carried out in a distributed way (several partitions of the dataset in parallel) and incrementally (aggregation of partial models), and due to its mathematical formulation, the data that is exchanged does not endanger the privacy of the users. This makes DAEF a valid method for edge computing and federated learning scenarios. The method has been evaluated and compared to traditional (iterative) deep autoencoders using seven real anomaly detection datasets, and their performance have been shown to be similar despite DAEF's faster training

SOPRENE: Assessment of the Spanish Armada’s Predictive Maintenance Tool for Naval Assets

Predictive maintenance has lately proved to be a useful tool for optimizing costs, performance and systems availability. Furthermore, the greater and more complex the system, the higher the benefit but also the less applied: Architectural, computational and complexity limitations have historically ballasted the adoption of predictive maintenance on the biggest systems. This has been especially true in military systems where the security and criticality of the operations do not accept uncertainty. This paper describes the work conducted in addressing these challenges, aiming to evaluate its applicability in a real scenario: It presents a specific design and development for an actual big and diverse ecosystem of equipment, proposing an semi-unsupervised predictive maintenance system. In addition, it depicts the solution deployment, test and technological adoption of real-world military operative environments and validates the applicability